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Dear Sir: 

We are in receipt of your letter dated September 9, 2005, copy enclosed, whereby you 
stated that the errors we submitted were made in accordance with the record in the Patent and 
Trademark Office. Please see enclosed a copy of our Amendment dated June 30, 2004, filed 
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I hereby certify that this correspondence is being transmitted by 
facsimile transmission to: Examiner Gary L. Solomon, Group Art 
Unit 2615, United States Patent and Trademark Office, Alexandria, 
VA 22313-1450, Facsimile No. 703-872-9306, on June 30, 2004. 

Wayne F. Reinke 
Attorney for Applicants 
Registration No.: 36,650 

Date of Signature: June 30, 2004. 

To: Mail Stop RCE 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 



AMENDMENT AND RESPONSE TO FINAL OFFICE ACTION 



Dear Sir: 



In response to a final Office Action with a mailing date of April 30, 2004, response to 
which is due by July 30, 2004 without extension, and which is being filed with a Request for 
Continued Examination, kindly amend the application as follows. 
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Amendment to the Specification: 

Amend the fourth full paragraph on page 2, lines 14-22, as follows: 

The present invention provides, in a first aspect, a method of augmenting an image of a 
scene. The method comprises a first computing unit obtaining a digital image of a scene, 
acquiring a digital elevation model of the scene, and registering the digital image with the digital 
elevation model to create a registered digital image. The digital elevation model comprises data 
for creating a three-dim ensional perspective model of the scene in two dimensions i o baood on 
e levation data corrooponding to information about the ocono . The method further comprises the 
first computing unit providing the registered digital image to a second computing unit coupled to 
the first computing unit by a communications network, and augmenting the registered digital 
image with at least some ef^information about the scene in response to input fi-om the second 
computing imit. 
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Amendment to the Abstract: 

Amend the first paragraph on page 27, Hnes 6-11, as follows: 

A fixed or mobile image capturing device, coupled to a first computer, is remotely 
controlled over a global computer network by a second computer, and obtains a digital image of 
a scene. The first computer acquires a digital elevation model of the scene, which is data for 
creating a three-dimensional perspective model of the scene in two dimensions b as e d on 
e levation data corr e sponding to information about the scono , and registers the image with the 
digital elevation model. The registered digital image is provided to the second computer over the 
global computer network. When a user at the second computer points to an area of the registered 
image, the first computer augments the image provided with at least some ef4he-information 
corresponding to the area pointed to. 
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Amendment to the Claims: 

In compliance with the Revised Amendment Foraiat, a complete listing of claims is 
provided herem. 

1 . (Currently Amended) A method of augmenting an image of a scene, comprising: 

obtaining by a first computing unit a digital image of a scene; 

acquiring by the first computing unit a digital elevation model of the scene, 
wherein the digital elevation model comprises data for creating a three-dimensional 
perspective m odel of the scene in two dimensions is bas e d on olovatinn dntn 
corr e sponding to information about tho sccno ; 

registering by the first computing unit the digital image with the digital elevation 
model to create a registered digital image; 

providing the registered digital image from the first computing unit to a second 
computing unit coupled to the first computing unit by a communications network; and 

augmenting the registered digital image provided to the second computing unit 
with at least some e#4he-information about the scene in response to input from the second 
computing unit. 

2. (Original) The method of claim 1, wherein obtaining the digital image comprises 
obtaining an image of the scene in real time with a fixed image capturing device. 

3. (Original) The method of claim 1, wherein obtaining the digital image comprises 
obtaining an image of the scene in real time with a mobile image capturing device. 

4. (Original) The method of claim 1, wherein obtaining the digital image comprises 
obtaining a preexisting image of the scene. 

5. (Original) The method of claim 1, wherein the obtaining comprises obtaining an 
analog image and digitizing the analog image. 



o , o 

-5- 0794.022 

6. (Original) The method of claim 1, wherein the acquiring comprises acquiring a 
preexisting digital elevation model. 

7. (Original) The method of claim 1, wherein the information comprises a location 
of at least one above-ground feature within the scene. 

8. (Original) The method of claim 1, wherein the information comprises a location 
of at least one underground feature within the scene. 

9. (Original) The method of claim 1, wherein the obtaining comprises obtaining an 
image of the scene from an image-capturing device, the method further comprising determining 
an orientation and a zoom factor for the image-capturing device. 

10. (Original) The method of claim 9, wherein the image-capturing device is fixed, 
and wherein the determining comprises reading positional and zoom factor information from the 
image-capturing device. 

1 1 . (Original) The method of claim 10, wherein the positional information comprises 
latitude, longitude and elevation above ground. 

12. (Original) The method of claim 9, wherein the image-capturing device is mobile, 
and wherein the determining comprises reading a zoom factor from the image-capturing device 
and obtaining the orientation from equipment positionally fixed relative to the image-capturing 
device. 



13. (Original) The method of claim 12, wherein the equipment comprises a global 
positioning system, a digital compass and a digital inclinometer. 
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14. (Original) The method of claim 1, further comprising rendering the registered 
digital image at the second computing unit to produce a rendered image, wherein the augmenting 
comprises: 

identifying at the second computing unit a point of interest in the rendered image; 

providing the identified point of interest from the second computing unit to the 
first computing imit; 

reverse projecting the identified point of interest at the first computing unit to 
determine coordinates therefor; and 

accessing the information at the first computing unit corresponding to the 
coordinates. 

15. (Original) The method of claim 14, wherein the augmenting further comprises 
overlaying the information corresponding to the coordinates on the registered digital image. 

16. (Original) The method of claim 15, wherein the information comprises textual 
cartographic data. 

17. (Original) The method of claim 15, wherein the information comprises graphical 
cartographic data. 

18. (Original) The method of claim 1, further comprising rendering the registered 
digital image at the second computing unit to create a displayed image, wherein the input 
comprises pointing by a user to at least one area of the displayed image. 



19. (Original) The method of claim 1, wherein the communications network 
comprises a global computer network. 



o o 

-7- - 0794.022 

20. (Currently Amended) A system for augmenting an image of a scene, comprising: 
a computing unit programmable for: 

obtaining a digital image of a scene; 

acquiring a digital elevation model of the scene, wherein the digital elevation 
model comprises data for creating a three-dimensional perspective model of the scene in 
two dimensions is based on e lovation datn cnrrnr.pnnHing tn infnrmnti^n nbont tho gcono ; 

registering the digital image with the digital elevation model to create a registered 
digital image; 

providing the registered digital image over a communications network; and 

augmenting the registered digital image with at least some efthe-information 
about the scene in response to input. 

21. (Original) The system of claim 20, further comprising a fixed image capturing 
device coupled to the computing unit for capturing an image of the scene in real time. 

22. (Original) The system of claim 20, further comprising a mobile image capturing 
device coupled to the computing unit for capturing an image of the scene in real time. 

23. (Original) The system of claim 20, further comprising a scanner for digitizing an 
analog image of the scene. 

24. (Original) The system of claim 20, wherein the acquiring comprises acquiring a 
preexisting digital elevation model. 

25. (Original) The system of claim 20, wherein the information comprises a location 
of at least one above-ground feature within the scene. 

26. (Original) The system of claim 20, wherein the information comprises a location 
of at least one underground feature within the scene. 
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27. (Original) The system of claim 20, further comprising an image-capturing device, 
the computing unit determining an orientation and a zoom factor for the image-capturing device. 

28. (Original) The system of claim 27, wherein the image-capturing device is fixed, 
and wherein the determining comprises reading positional and zoom factor information from the 
image-capturing device. 

29. (Original) The system of claim 28, wherein the positional information comprises 
latitude, longitude and elevation above ground. 

30. (Original) The system of claim 27, further comprising orientation equipment 
positionally fixed relative to the image-capturing device, wherein the image-capturing device is 
mobile, and wherein the determining comprises reading a zoom factor from the image-capturing 
device. 

3 1 . (Original) The system of claim 30, wherein the orientation equipment comprises a 
global positioning system, a digital compass and a digital inclinometer. 

32. (Original) The system of claim 20, further comprising another computing unit 
coupled to the computing unit via the communications network, the another computing unit 
comprising: a display for rendering the registered digital image to produce a rendered image; 
and a pointing device for identifying a point of interest in the rendered image; wherein the 
augmenting comprises reverse projecting the identified point of interest to determine coordinates 
therefor, and accessing the information corresponding to the coordinates. 

33. (Original) The system of claim 32, wherein the augmenting further comprises 
overlaying the information corresponding to the coordinates on the rendered image. 

34. (Original) The system of claim 33, wherein the information comprises textual 
cartographic data. 

35. (Original) The system of claim 33, wherein the information comprises graphical 
cartographic data. 
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36. (Original) The system of claim 32, wherein the communications network 
comprises a global computer network. 

37. (Currently amended) At least one program storage device readable by a machine, 
tangibly embodying at least one program of instructions executable by the machine to perform a 
method of augmenting an image of a scene, the method comprising: 

obtaining by a first computing unit a digital image of a scene; 

acquiring by the first computing unit a digital elevation model of the scene, 
wherein the digital elevation model comprises data for creating a three-dimensional 
perspective model of the scene in two dimensions is bas e d on elevation data 
corr e sponding to information about th e sccno ; 

registering by the first computing unit the digital image with the digital elevation 
model to create a registered digital image; 

providing the registered digital image from the first computing unit to a second 
computing unit coupled to the first computing unit by a communications network; and 

augmenting the registered digital image provided to the second computing unit 
with at least some ef^the-information about the scene in response to input fi-om the second 
computing unit. 

38. (Original) The at least one program storage device of claim 37, wherein obtaining 
the digital image comprises obtaining a digital image of the scene in real time fi*om a fixed 
digital image capturing device. 

39. (Original) The at least one program storage device of claim 37, wherein obtaining 
the digital image comprises obtaining a digital image of the scene in real time fi-om a mobile 
digital image capturing device. 



40. (Original) The at least one program storage device of claim 37, wherein obtaining 
the digital image comprises obtaining a preexisting digital image of the scene. 
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41 . (Original) The at least one program storage device of claim 37, wherein acquiring 
the digital elevation model comprises acquiring a preexisting digital elevation model. 

42. (Original) The at least one program storage device of claim 37, wherein the 
information comprises a location of at least one above-ground feature within the scene. 

43. (Original) The at least one program storage device of claim 37, wherein the 
information comprises a location of at least one underground feature within the scene. 

44. (Original) The at least one program storage device of claim 37, wherein the 
obtaining comprises obtaining the digital image from a digital image-capturing device, the 
method further comprising determining an orientation and a zoom factor for the digital image- 
capturing device. 

45. (Original) The at least one program storage device of claim 44, wherein the digital 
image-capturing device is fixed, and wherein the determining comprises reading positional and 
zoom factor information from the digital image-capturing device. 

46. (Currently Amended) The at least one program storage device of claim [[3711 45. 
wherein the positional information comprises latitude, longitude and elevation above ground. 

47. (Original) The at least one program storage device of claim 44, wherein the digital 
image-capturing device is mobile, and wherein the determining comprises reading a zoom factor 
from the digital image-capturing device and obtaining the orientation from equipment 
positionally fixed relative to the digital image-capturing device. 

48. (Original) The at least one program storage device of claim 47, wherein the 
equipment comprises a global positioning system, a digital compass and a digital inclinometer. 
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49. (Original) The at least one program storage device of claim 37, further comprising 
rendering the registered digital image at the second computing unit to produce a rendered image, 
wherein the augmenting comprises: 

identifying at the second computing unit a point of interest in the rendered image; 

providing the identified point of interest from the second computing unit to the 
first computing unit; 

reverse projecting the identified point of interest at the first computing unit to 
determine coordinates therefor; and 

accessing the information at the first computing unit corresponding to the 
coordinates. 

50. (Original) The at least one program storage device of claim 49, wherein the 
augmenting further comprises overlaying the information corresponding to the coordinates on 
the registered digital image. 

51. (Original) The at least one program storage device of claim 50, wherein the 
information comprises textual cartographic data. 

52. (Original) The at least one program storage device of claim 50, wherein the 
information comprises graphical cartographic data. 



53. (Original) The at least one program storage device of claim 37, wherein the 
communications network comprises a global computer network. 
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54. (New) A system of augmenting an image of a scene, comprising: 

means for obtaining by a first computing unit a digital image of a scene; 

means for acquiring by the first computing unit a digital elevation model of the 
scene, wherein the digital elevation model comprises data for creating a three- 
dimensional perspective model of the scene in two dimensions; 

means for registering by the first computing unit the digital image with the digital 
elevation model to create a registered digital image; 

means for providing the registered digital image from the first computing unit to a 
second computing unit coupled to the first computing imit by a communications network; 
and 



means for augmenting the registered digital image provided to the second 
computing unit with at least some information about the scene in response to input from 
the second computing unit. 
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It is certified that error appears in the above-identified patent and that said Letters Patent 
IS hereby corrected as shown below: 

Title Page: 
Abstract: 

Line 7, delete the following wording: "based on data corresponding" 
Specification: 

Col. 1, line 64, insert the word -perspective- after the words "three-dimensional" 

Claims: 
Claim 20 

Coi. 12, line 10, delete the word "of at the beginning of the line 
Claim 37 

Col. 13, line 14, delete the word "of after the word "some" 
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